Machine Learning to Assess the Prognostic Utility of Radiomic Features for In-hospital COVID-19 Mortality (preprint)

As portable chest X-rays are an efficient means of triaging emergent cases, their increased use has raised the question as to whether imaging carries additional prognostic utility for survival among patients with COVID-19. This study assessed the importance of known risk factors on in-hospital mortality and to investigate the predictive utility of radiomic texture features using various machine learning approaches. We detected incremental improvements in survival prognostication utilizing texture features derived from emergent chest X-rays, particularly among older patients or those with higher comorbidity burden. Important features included age, oxygen saturation, blood pressure, and certain comorbid conditions, as well as image features related to the intensity and variability of the pixel distribution. Thus, widely available chest X-rays, in conjunction with clinical information, may be predictive of survival outcomes of patients with COVID-19, especially older, sicker patients, and can aid in disease management by providing additional information.

Portable Breath-Based Volatile Organic Compound Monitoring for the Detection of COVID-19: Challenges of Emerging Variants (preprint)

Importance Breath analysis has been explored as a noninvasive means to detect COVID19. However, the impact of the emerging variants such as Omicron on the exhaled breath profile and hence the accuracy of breath analysis is unknown. Objective To evaluate the diagnostic accuracies of breath analysis on detecting COVID19 patients in periods where Delta and Omicron were most prevalent. Design, Setting, and Participants A convenience cohort of patients testing positive and negative for COVID19 using reverse transcriptase polymerase chain reaction (RTPCR) were studied and included 167 COVID and nonCOVID patients recruited between April 2021 and May 2022, which covers the period when Delta (and other variants prior to Delta) was the dominant variant (April to December 2021) and when Omicron was the dominant variant (January to May 2022). The breath from those patients were collected and analyzed for volatile organic compounds (VOCs) with a newly developed portable gas chromatography based breath analyzer. Diagnostic patterns and algorithms were developed. Results A total of 205 breath samples were analyzed from 167 COVID and nonCOVID patients. The RTPCR was conducted within 18 hours of the breath analysis to confirm the COVID status of the patients. Among 94 COVID positive samples, 41 samples were collected from the patients in 2021 who were assumed to be infected by the Delta variant (or other variants occurring in 2021) and 53 samples from the patients in 2022 who were assumed to be infected by the Omicron variant (BA.1 and BA.2). Breath analysis using a set of 4 VOC biomarkers was able to distinguish between COVID (Delta and other variants in 2021) and nonCOVID with an overall accuracy of 94.7%. However, the accuracy dropped significantly to 82.1% when the same set of biomarkers were applied to the Omicron variant with and 21 out of 53 COVID positive being misidentified. A new set of 4 VOC biomarkers were found to distinguish the Omicron variant and non-COVID, which yielded an overall accuracy of 90.9%. Breath analysis was also found to be able to distinguish between COVID (for all the variants occurring between April 2021 and May 2022) and nonCOVID with an overall accuracy of 90.2%, and between the Omicron variant and the earlier variants (Delta and other variants occurring in 2021) with an overall accuracy of 91.5%. Conclusions and Relevance Breath analysis of VOCs using point of care gas chromatography may be a promising diagnostic modality for detection of COVID and similar diseases that result in VOC production. However, similar to other diagnostic modalities such as rapid antigen testing, challenges are posed by the dynamic emergence of viral variants. The results of this study warrant additional investment and evaluation on how to overcome these challenges and to exploit breath analysis to improve the diagnosis and care of patients.

Comprehensive Evaluation of COVID-19 Patient Short- and Long-term Outcomes: Disparities in Healthcare Utilization and Post-Hospitalization Outcomes (preprint)

BackgroundUnderstanding risk factors for short- and long-term COVID-19 outcomes have implications for current guidelines and practice. We study whether early identified risk factors for COVID-19 persist one year later and through varying disease progression trajectories. MethodsThis was a retrospective study of 6,731 COVID-19 patients presenting to Michigan Medicine between March 10, 2020 and March 10, 2021. We describe disease progression trajectories from diagnosis to potential hospital admission, discharge, readmission, or death. Outcomes pertained to all patients: rate of medical encounters, hospitalization-free survival, and overall survival, and hospitalized patients: discharge versus in-hospital death and readmission. Risk factors included patient age, sex, race, body mass index, and 29 comorbidity conditions. ResultsYounger, non-Black patients utilized healthcare resources at higher rates, while older, male, and Black patients had higher rates of hospitalization and mortality. Diabetes with complications, coagulopathy, fluid and electrolyte disorders, and blood loss anemia were risk factors for these outcomes. Diabetes with complications, coagulopathy, fluid and electrolyte disorders, and blood loss were associated with lower discharge and higher inpatient mortality rates. ConclusionsThis study found differences in healthcare utilization and adverse COVID-19 outcomes, as well as differing risk factors for short- and long-term outcomes throughout disease progression. These findings may inform providers in emergency departments or critical care settings of treatment priorities, empower healthcare stakeholders with effective disease management strategies, and aid health policy makers in optimizing allocations of medical resources.

Validation and comparison of PICTURE analytic and Epic Deterioration Index for COVID-19 (preprint)

Introduction The 2019 coronavirus (COVID-19) has led to unprecedented strain on healthcare facilities across the United States. Accurately identifying patients at an increased risk of deterioration may help hospitals manage their resources while improving the quality of patient care. Here we present the results of an analytical model, PICTURE (Predicting Intensive Care Transfers and other UnfoReseen Events), to identify patients at a high risk for imminent intensive care unit (ICU) transfer, respiratory failure, or death with the intention to improve prediction of deterioration due to COVID-19. We compare PICTURE to the Epic Deterioration Index (EDI), a widespread system which has recently been assessed for use to triage COVID-19 patients. Methods The PICTURE model was trained and validated on a cohort of hospitalized non-COVID-19 patients using electronic health record data from 2014-2018. It was then applied to two hold-out test sets: non-COVID-19 patients from 2019 and patients testing positive for COVID-19 in 2020. PICTURE results were aligned to the EDI for head-to-head comparison via Area Under the Receiver Operator Curve (AUROC) and Area Under the Precision Recall Curve (AUPRC). We compared the models' ability to predict an adverse event (defined as ICU transfer, mechanical ventilation use, or death) at two levels of granularity: (1) maximum score across an encounter with a minimum lead time before the first adverse event and (2) predictions at every observation with instances in the last 24 hours before the adverse event labeled as positive. PICTURE and the EDI were also compared on the encounter level using different lead times extending out to 24 hours. Shapley values were used to provide explanations for PICTURE predictions. Results PICTURE successfully delineated between high- and low-risk patients and consistently outperformed the EDI in both of our cohorts. In non-COVID-19 patients, PICTURE achieved an AUROC (95% CI) of 0.819 (0.805 - 0.834) and AUPRC of 0.109 (0.089 - 0.125) on the observation level, compared to the EDI AUROC of 0.762 (0.746 - 0.780) and AUPRC of 0.077 (0.062 - 0.090). On COVID-19 positive patients, PICTURE achieved an AUROC of 0.828 (0.794 - 0.869) and AUPRC of 0.160 (0.089 - 0.199), while the EDI scored an AUROC of 0.792 (0.754 - 0.835) and AUPRC of 0.131 (0.092 - 0.159). The most important variables influencing PICTURE predictions in the COVID-19 cohort were a rapid respiratory rate, a high level of oxygen support, low oxygen saturation, and impaired mental status (Glasgow coma score). Conclusion The PICTURE model is more accurate in predicting adverse patient outcomes for both general ward patients and COVID-19 positive patients in our cohorts compared to the EDI. The ability to consistently anticipate these events may be especially valuable when considering a potential incipient second wave of COVID-19 infections. PICTURE also has the ability to explain individual predictions to clinicians by ranking the most important features for a prediction. The generalizability of the model will require testing in other health care systems for validation.

Validating a Widely Implemented Deterioration Index Model Among Hospitalized COVID-19 Patients (preprint)

IntroductionThe Epic Deterioration Index (EDI) is a proprietary prediction model implemented in over 100 U.S. hospitals that was widely used to support medical decision-making during the COVID-19 pandemic. The EDI has not been independently evaluated, and other proprietary models have been shown to be biased against vulnerable populations. MethodsWe studied adult patients admitted with COVID-19 to non-ICU care at a large academic medical center from March 9 through May 20, 2020. We used the EDI, calculated at 15-minute intervals, to predict a composite outcome of ICU-level care, mechanical ventilation, or in-hospital death. In a subset of patients hospitalized for at least 48 hours, we also evaluated the ability of the EDI to identify patients at low risk of experiencing this composite outcome during their remaining hospitalization. ResultsAmong 392 COVID-19 hospitalizations meeting inclusion criteria, 103 (26%) met the composite outcome. Median age of the cohort was 64 (IQR 53-75) with 168 (43%) African Americans and 169 (43%) women. Area under the receiver-operating-characteristic curve (AUC) of the EDI was 0.79 (95% CI 0.74-0.84). EDI predictions did not differ by race or sex. When exploring clinically-relevant thresholds of the EDI, we found patients who met or exceeded an EDI of 68.8 made up 14% of the study cohort and had a 74% probability of experiencing the composite outcome during their hospitalization with a median lead time of 24 hours from when this threshold was first exceeded. Among the 286 patients hospitalized for at least 48 hours who had not experienced the composite outcome, 14 (13%) never exceeded an EDI of 37.9, with a negative predictive value of 90% and a sensitivity above this threshold of 91%. ConclusionWe found the EDI identifies small subsets of high- and low-risk COVID-19 patients with fair discrimination. We did not find evidence of bias by race or sex. These findings highlight the importance of independent evaluation of proprietary models before widespread operational use among COVID-19 patients.